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ABSTRACT 

Intended for policymakers, admnistrators, and others 
involved in determining how the efficiency and effectiveness of 
technical and further education (TAFE) should be measured, this 
monograph contains the following sections: (1) "Introduction," 
including a explanation of the origins of the push for performance 
indicators and the re ponse to it; (2) "A Case foi Performance 
Indicators? Some Conceptual Issues," which discusses 
cost-ef fectiveneSvS and the role of performance indicators; (3) "What 
Are Performance Indicators? Some Definitions," including the 
relationship between performance indicators and program evaluation 
and validation; (4) "Four Factors to be Considered in Developing 
Performance Indicators," which explains the political, financial, 
administrative, and educational factors; (5) "Some Performance 
Indicators," with examples of those that indicate effectiveness and 
those that indicate efficiency; (6) "Performance Indicators — Some 
Issues and Concerns"; and (7) "Some Conclusions." A list of 44 
references concludes the document. (CML) 
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INTRODUCTION 



Whilst educational accountability Is not a new phenomenon In 
TAFE, the push towards the development of Indicators of 
performance as variables within an accountability mechanism has 
gained Impetus In Australia over recent years. Much of this 
Impetus has come from demands by stakeholders In the educational 
process that better use be made of scarce and often diminishing 
resources. Better use usually implies that both the efficiency 
and effectiveness of provision be Improved. Performance 
Indicators have been or are being developed by TAFE systems 
throughout Australia. 

Performance Indicators In an educational context are broadly 
concerned with measuring educational outcomes and are 
reflections of efficiency and effectiveness; In other words, 
performance Indicators are those factors which will provide a 
perception of what Is actually going on and how well things are 
working. Examples of Indicators Include: 

graduate success rate ; 

student and Industry satisfaction with program provision; 

participation rate according to demographic group; 

recurrent cost per subject per student; and 

average teaching hours per full-time TAFE lecturer. 

TH£ COMMONUEALTH'S PUSH FOR PERFORMANCE INDICATORS 

Skills for Australia , a discussion paper circulated by the Hon. 
John Oawklns and the Hon. Clyde Holding, has suggested the need 
for fiscal constraint as well as the need to Improve both the 
efficiency and effectiveness of education and training. This 
paper is worth quoting at length. It points out that: 

The Government recognises that there a number of 
approaches which can be taken to achieve the desired 
Increases In the quality and flexibility of our labour 
force skills. Some of these . . . will Involve the 
developopent of new structures and programs of action. 
Others will rely heavily on the enhancement and more 
effective use of existing structures. 
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The best possible result must be achieved with the 
resources which can be aade available. As In other 
Indiistrles served by our education and training 
system, there Is considerable scope to Improve the 
outputs from our present stock of education and 
training resources. Given that salary costs make up 
about 80% of total recurrent budgets for education and 
training, the productivity of staffing resources Is 
one Issue that requires attention. 

Special attention will also be paid to the relative 
efficiency and value of the various approaches taken 
to achieve program objectives. Accordingly, plans 
for the systematic monitoring and evaluation of each 
program will be developed. The l^>act of programs will 
be assessed to determine their continued relevance and 
priority, and to ensure that desired outcomes are 
achieved In the most cost-effective manner, (my 
emphasis) A flexible approach to the achievement of 
the Government's objectives will thus be maintained, 
and programs modified In the light of effectiveness 
and efficiency considerations. 

(Skills for Australia. 1987 p. 17*18) 

Moreover the paper suggests that with respect to the current 
Commonwealth funding arrangements there are a number of 
significant limitations In TAPE, especially In the recurrent 
expenditure area. These Include: 

an absence of clear program goals and targets; 

a heavy emphasis on the funding of educational Inputs 
rather than the achievement of specified outcomes; 

for the most part, the lack of a direct relationship to 
wider economic. Industry development or labour market 
objectives ; 

the lack of Inducements to greater public and private 
sector co-operation In the delivery of training and the 
sharing of training costs; 

the absence of any substantial evaluation of program 
effectiveness . 

(Skills for Australia. 1987 p. 33) 
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The Skllla f or Australia document (1987) goes on to point out 
that the Review of TAFE Funding report (May 1986) concluded that 
there was a need tur "considerable restructuring of Commonwealth 
grants for TAFE and the arrangements \inder which they are 
provided In order to s^rve more effectively an outcomes- oriented 
approach with effective accour.tability" ( Skills for Australia . 
1987. p. 33). 

The federal government's recent White Paper on higher education 
supports the development of a system of performance indicators 
and intends to develop funding arrangements: 

. . . that take into account a range of output, 
quality and performance measures . . . 

(Higher Education, 1988 p. 85) 

These performance measures could be based on mutually agreed 
goals. The Australian Vice-Chancellor's Committee (AVCC) and 
the Australian Oommittee of Directors and Principals in Advanced 
Education (ACDP) are attempting to define: 

a comprehensive and workable set of performance 
indicators for general application in higher education 
institutions. 

(Higher Education, 1988 p. 86) 

The preliminary report of the Joint AVCC/ACDP working party Is 
cited in the Bibliography on page 49. 

From the fedc'ral perspective there is clearly a strong 
inducement for TAFE and other tertiary systems and institutions 
receiving federal monies to develop appropriate and useful 
indicators of performance. 



INITIATIVES AT THE STATE LEVEL 

John ^ite, the TAFE National Centre's first Senior Fellow, 
examined the issue of educational accountability and the need 
for comprehensive evaluation in TAFE in order to provide an 
extension and reinforcement for an educational management 
process based on effective planning (White 1987) . His report 
provides a useful overview to the whole question of using 
performance indicators in TAFE at the state level. 

In New South Wales the Annual Renorts (^Departments) Act. 1985 
requires that annual reports of particular government 
departments and statutory bodies include qualitative and 
quantitative measures of performance which evaluate the 
appropriateness, efficiency and effectiveness of progress as 
well as information about client reaction to the services 




provided. The nemorandua froa the Premier of Nev South Vales to 
hie Mlnliters entitled "Management Performance Review (16 April 
1986)" states: 

Reports vlll be narrative, but as required by 
Regulations being drafted under the Annual Reports 
Acts, will, where practicable. Include qualitative and 
quantitative measures and Indicators of performance 
showing levels of efficiency and effectiveness. 

Emphasis should be on the development of performance 
Indicators which relate to or explain what Is being 
achieved by the programs. These performance Indicators 
should make possible comparison with past results and 
planned achievements. 

(Wells, 1986 p,2-3) 

In addition White (19L7) has noted that the NSW legislation 
requli^s that an account of the major Issues and problems 
encountered during the year of reporting be provided. 

In Western Australia the Financial Administra tion and Audit Act:. 
1986 requires every government department and Instrumentality to 
produce an annual report containing performance Indicators 
(Henderson, 1987). This legislation was based on that used In 
New South Wales. 

As part of the Savage Initiatives all government departments and 
sub -departments In Queensland are required to establish: 

a strategic plan; 

an on- going planning process; 

performance Indicators; and 

a program management structure (Newell 1987) . 

Most other state/territory TAFE authorities use performance 
Indicators at least to some extent. For example In South 
Australia the appointment of a new Director-General has added 
Impetus to the development of performance measures In that 
state. Victorian TAFE has developed a performance Improvement 
plan. There are also negotiated performance agreements between 
the State Training Board and Individual colleges. 

SOKE RESPONSES BY TAFE AT THE NATIONAL LEVEL 

Early In 1988 a 2-day meeting was held In Melbourne which 
gathered together Individuals from a number of states and the 
TAFE National Centre who had an Interest In the development and 
use of performance Indicators. Their deliberations produced a 
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report which was submitted to the Conference of TAFE Directors 
(TAPE Perfornuince Indicators Plaining Group 1988). The report 
has also been considered at the Australian Committee on TAFE 
Curriculum. As a result of this report a national working party 
was set up to look at national data requirements for TAFE. A 
meeting was held In early August convened by John Swan, the 
General Manager, Vocational Education and Training, Queensland. 
Tnls meeting resolved that: 

the report of the nationally-based working group on 
performance Indicators which xuet In Melbourne In February 1 
and 2 be endorsed, with some modifications; and 

that the working group should reconvene to refine 
definitions of chosen Indicators and to develop 
methodologies for collecting the Information. 

Clearly there Is considerable Interest In performance Indicators 
In TAFE In Australia. 

THE AIMS OF THIS MONOGRAPH 

This present monograph has a number of purposes. Given the 
Increasing Interest In the measurement of educational 
performance it seems timely to produce a monograph which will 
examine: 

why there may or may not be a case for developing and using 
performance Indicators; 

what performance Indicators are; 

the context In which they are or will be used; 

some of the commonly-used measures; and 

some of the Issues and questions which arise when 
performance Indicators are being considered or are already 
In use . 

Finally the monograph will try to draw some useful and valuable 
conclusions . 



A CASS FOR PBRFOBNAHCB IHD7CATOCLS7 - SOKE CONTEmiAL ISSUES 



THE NOBAL DINBNSIOM OF TAFE 

Technical and f\u:th>dr education la a aaxvlca industry. It Is 
concamad with providing learning opportunities for those In 
Industry an . coimerco, lor tradespeople and technicians - Indeed 
a whole range of Important occupational areas vlch a vide 
variety of training needs. In addition, TAFE services society's 
further education needs by providing enrlchaent progran^s and 
post-initial training. Furthemore, TAFE provides a means c2 
access to education for those Isolated by distance or by 
personal and physical hanolcaps. Finally, it attempts to cater 
for the needs of significant minority groups such as migrants 
and Aborlgir^l people. Therefore, while TAFE has the Important 
role in the mainstream of education and training, it has an 
important moral role as well. TAFE represents an Important 
mechanism by which society exercises Its duty and role to 
provide its members with the best education and the widest range 
of opportunities possible. For sc!*e time TAFE has been amongst 
the vanguard of those bodies attempting to redress some of the 
existing inequalities as well as providing educational services 
to those who might otherwise be denied these opportunities. TAFE 
attempts to enhance peoples' options and choices. One of TAFE's 
major problems hovever, is to determine priorltli** from the 
competing demands made upon It by various sectors in society. 

THE FRAGHATIC DI!4ENSI(W OF TAFE 

The way in which TAFE goes about providing options and choices 
ranging from the individualised approach which customises a 
program to suit a particular circumstance, to the package deal 
which (as Theodossin 1987 points out) is th^ down- marKet end of 
the educational business. The down-market end is cliaracterised 
by set curricula and off-the-peg materials. Modifications and 
extras are not allowed or, at least, actively discouraged. At 
best, they are available at extra cost. As one ^oves 
progressively up-market the programs are increasingly tailored 
to meet the individual needs of the c**ient. The provision of 
educational goods and services attracts costs and it is the 
client's willingness to pay or the system's willingness (or 
ability) to provide those services which dictate what an 
educational package will finally cost the public purse. 

Moreover, relatively large and complex systems like TAFE do not 
change precipitously, although presently there are significant 
changes going on, or mooted, in a number of key areas of the 
Australian economy. Providers are constrained by state-based 
and even national curricula, industry and occupational 
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structures, Industrial awards and recognised career pathways. 
Change, when It occurs, Is generally slow and sometimes 
Incremental. This may suit the systems because they have often 
Invest© 1 considerable time and effort (and therefore money) In 
developing programs. Wholesale and rapid changes may mean that 
desirable economies of scale are not achieved. If materials date 
rapidly resources have to be Invested to provide more up-to-date 
materials. The cost of doing this needs to be Justified. TAFE 
authorities, though, recognise that they have to be relevant. 

Another dimension of TAFE Is therefore pragmatic. It recognises 
that Its programs cost money and that those who control 
educational funds are being held more and more accountable for 
their use. In short, TAFE has to be, and be seen to be, relevant 
and both efficient and effective. It has to make the best use of 
the resources allocated to It and may have to Justify the 
expenditure of the funds It has received. It may also have to 
Justify Its claims for additional funds to enhance or restructure 
existing services, or to develop new services. The development of 
new services may mean that exiscing services In certain areas 
have to be reduced or curtailed altogether. This may lead to hard 
decisions being made about which services can be reduced or 
discontinued In order to service areas of expanding need. The 
pragmatic dimension Is concerned with accountability and with 
ensuring efficiency and effectiveness. Nevertheless the moral 
dimension cannot be Ignored In the decision-making process. Hence 
programs which could never be Justified In pragmatic terms still 
need to be developed or maintained because of their moral 
significance. In pragmatic terms they should be as efficient and 
effective as possible, but their moral Importance precludes them 
being Judged by generally- held criteria. To ensure that the 
moral and pragmatic dimensions are balanced properly, any 
d'^clslons made should be based on informed Judgements. This Is 
where performance Indicators come In. 

THE NOTIOW OF COST- EFFECTIVENESS 

Being cost-effective Is believed to be a desirable educational 
outcome, given the present concerns about financial restraint and 
the creation of additional learning capacity at minimum cost 
(particularly to governments). Some have suggested that the 
drive towards improving cost-effectiveness is efficiency- 
motivated (for example, Franchak 1981). The real context of 
cost-effectiveness seems, to me, to be the measurement of 
performance, or outcomes, in relation to cost. Nevertheless the 
proper balancing of the moral and pragmatic dimensions of TAFE's 
provision of services cannot be Ignored. Cost-effectiveness is 
therefore part of our increasing concern for developing and using 
indicators of performance to measure the outcomes of educational 
systems such as TAFE and their component parts. Performance 
indicators can therefore be used as management tools. 



As Nevell (1987) points out, there are two main criteria that 
have to be considered In assessing TAFE's performance In the 
provision of Its services, namely: 

Its responsiveness to comnunlty and Indxistry needs In the 
development and provision of adequate and timely programs; 
and 

operational efficiency and effectiveness. 

Nevell 's criteria embrace both the moral and pragmatic 
dimensions . 

THE ROLE OF PERFORMANCE INDICATORS 

Why use performance Indicators? The balancing of the moral and 
the pragmatic dimensions clearly Indicate that the "benchmarks" 
which might be used to compare performances would shift according 
to the position of the appropriate balance point. Like programs 
might be compared with like, but comparing the performance of an 
access program for Aboriginal people In remote areas with the 
performance of a major city-based trade course would clearly have 
problems. 

Performance Indicators relate to the way In which TAPE does what 
It does. Most of TAPE' s activity Is based on Identified or 
"perceived" needs. These needs generate "objectives". Objectives 
lead to "strategies" being put In place. These strategies have 
"outcomes" (Figure 1). 

This Is a linear planning process. Naturally In "real life" this 
linear process has a number of feedback loops and I will describe 
some of these shortly. 




Figure 1 
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Onct som« sort of strategy Is In place the development of 
appropriate 4ctlvltles can occur. The strategy night, for 
exanple, be a curriculum document or a plan to provide training 
and retraining for an Industry. Thus strategies have "activities" 
or "doing" things associated with them. These activities have 
"Inputs" and give rise to "outputs" (Figure 2). Inputs are 
defined hy BlA.ch and Latcham (1985) as all the variables entering 
tho system - for example: 



students 



staff 



their knowledge, skills, aptitudes, attitudes, 
motivations and expectations at rntry; 



academic, administrative, technical 
competencies , experience , attitudes , 
motivations and expectations; 



their 



money 



the funds committed to or earned by the 
system; and 



physical - 
facilities 



buildings, books, equipment etc. 



Oucputs are defined (Henderson, 1987) as events, objects or 
services generated by program activities. He defines outcomes as 
the effect, the Impact, which the outputs are Intended to 
exercise on the original need or problem. Thus outputs might be 
considered as Indicators of the extent to which some of the 
desired outcomes have been achieved, given the original need or 
problem. Outputs and outcomes are therefore related. 



Planning 



Doing 





Figure 2 
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Th« rtlatiotuihip batveen inputs and outputs is used in the 
developnent of efficiency measures. Efficiency aeasures are 
usually relatively easy to quantify and hence are useful as gross 
conparative aeasures. The relationship between objectives and 
needs helps to detemine the a ppropriataness of an activity or 
strategy. Finally, the relationship between the outcomes and 
objectives gives cn indication of program effectiveness (Figure 
3). Ideally, performance indicators are concerned with measuring 
the efficiency, effectiveness and appropriateness of programs, 
activities or strategies. 



Planning Doing 




Appropriateness 




Efficiency 



figure 3 



11 



17 



SOME CONCERNS ABOUT PERFOSNANCB INDICATORS 



When needs and outcomes are discussed It must be clearly 
recognised that: 

there are real and perceived needs . The success of any 
activity, or strategy, la tied up with the extent to which 
the perceived needs on which It was based match the real 
needs the client group had. In essence a program could 
achieve Its outcomes, but If the perceived needs on which It 
was based originally were Inappropriate, or had changed. It 
might be argued that the program had not performed well; and 

there are Intended and unintended outcomes. The Intended 
outcomes are clearly based on the objectives of the strategy 
which, in turn, were based on the perceived needs. Thus, 
attention is focused on the Intended outcomes and the extent 
to which they were achieved. (This gives a measure of 
effectiveness.) However if the program, through its 
activities produces other outcomes, what thea? To what 
extent do they contribute to Judgements about effectiveness? 
These unintended outcomes could, after all, be good or bad, 
but the system which monitors performance may take little if 
any account of these. It is likely that data relevant to 
these outcomes will not even be collected. ThuSi how does 
"the system" cope with an Important but unintended outcome? 

TAPE systems as a rule dedicate considerable effort to the 
identification of needs and the prcdutlon of objectives - and to 
the planning and designing of activities. To a lesser extent it 
collects information about Inputs and outputs. Little, if any 
emphasis is placed on the effective and comprehensive measurement 
of educational outcomes, yet good effectiveness measures are 
needed if TAFE's performance is to be Judged validly. 

Education systems often have a tendency to view accountability as 
a method of punishment. TAPE is no exception to this. As 
Henderson (1987) points out, there is no doubt that performance 
indicators can be used punltlvely. Amongst the issues and 
concerns raised later in this paper will be the number and type 
of indicators chosen, who uses them and how they are used. 
Nevertheless there are also many positive reasons for using 
performance indicators. 

PERFORMANCE INDICATORS - SOKE POSITIVES 

Theoretically at least, the arguments supporting the development 
and use of performance Indicators is a compelling one. Indicators 
of performance can provide valuable planning and management 
tools. Therefore they are defensible in pragmatic terms. 
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In addition to thalr "'pragnatlc* value as aanagemant tools they 
are morally defensible as well. TAFB Is. and should continue to 
be, accountable for what It does. Govemaents and other 
Important client and Interest groups have a right to know the 
quaMty of the services provided, that la, how well these 
services measure up. Performance Indicators can provide 
measures of the extent to which the TAPE systems and their 
components are responsive to society's needs. For example: 

Have the short- to medium- term needs of the economy been 
addressed by TAFE's activities? 

Has the relevant TAPE system taken account of government 
strategies to enhance social Justice? 

These, and many other questions can be directed at TAFE. 
Questions can also be asked about the quality of the programs 
provided. Questions might also be asked about the mechanisms 
TAFE uses to Improve the quality of Its program content, Its 
processes and Its products. Failure to provide satisfactory 
answers can, at worst, leave TAFE open to a potential barrage of 
Innuendo and unsubstantiated assertion which imdermlnes TAFE's 
credibility In the eyes of Its clients, Including the government 
and the general community. If performance Information Is 
readily available such situations should not arise. 

In summary, performance Indicators may be used In a positive 
way: 

for Improvement -orientated decision-making; 

as a management tool to maintain or enhance the efficiency 
of the resources and activities for which they are 
responsible; 

for diagnostic purposes - In that they allow Identification 
of strengths and weaknesses which, following analysis, 
would allow remediation measures to be taken to Improve 
effectiveness ; 

to ensure that TAFE Xl meeting the needs of Important 
client groups (Including the rolu It is asked to play In 
achieving greater social equity and Improving Individual 
peoples ' opportunities . ) 

Essentially I am asserting that, in theory at least, performance 
indicators are a valuable and necessary management, decision- 
making and diagnostic tcol. They have a planning role. They tlso 
have an accountability role. Finally they can be used as a 
mechanism of q\iality assurance. This relates, in part, to their 
management role, but allows those at all levels within TAFE to 
participate in the diagnostic, decision-making and improvement 
process. 
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However, performance Indicators can succeed or fail In the 
translation of a theoretically sound concept Into practice. 
This monograph will now consider what performance Indicators 
are, and will deal with factors affecting their use. It will 
also consider the various Issues and concerns raised about then 
and the way they are used In practice. 



14 



zo 



WHAT ARE PERFORMANCE INDICATORS? • SOKE DEFINITIONS 



PERFORMANCE INDICATORS DEFINED 

As I have already suggested, performance Indicators are broadly 
concerned with measuring educational outcomes and are 
reflections of efficiency and effect Iven^ass; In other words, 
performance Indicators are factors which will provide a 
perception of what Is actually going on, and how well things are 
working. They are Indicators of how well, and how efficiently, 
programs or other system components art achieving the desired 
outcoires. Theodossln (1987) polrits out that. In using 
performance Indicators: 

We are seeking Indications, characteristics, pointers, 
tokens, signals, symbols, something partial which 
reflects a larger whole, performance Indicators which 
serve a surrogate or proxy function. 

(Theodossln, 1987 p. 9) 
Henderson (1987) has defined performance Indicators as: 

. . . indicators of the extent to which programs are 
achieving the leslred results. They should define the 
measurement oi a piece of Important and useful 
Information about the performance of a programme 
expressed as an Index, rate or other comparison which 
Is monitored at regular Intervals and Is compared with 
a benchmark. 

(Henderson, 1987 p. 3) 

Performance Indicators are therefore substitutes for things 
which cannot be assessed directly. Moreover, I have already 
argued, that system performance is affected by the extent to 
which the perceived needs correctly reflect the real needs. 
Therefore any system of assessing performance must ultimately 
seek to assess the appropriateness of its strategies and 
activities. 

Henderson (1987) points out that the Western Australian act 
requires that indicators be: 

relevmt - they must be logically rBlated to user's needs 
and be based on objectives which clearly communicate what 
is to be measured; 
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verifiable - appropriately qualified Individuals working 
Independently should arrive at essentially similar 
conclusions or results; 

frmm tram blae - data are Impartially gathered, the 
statistical treatment Is free fro.i bias, and the dat- are 
Impartially reported; and 

QMntlfl i^'^** - measurable against pre-determlned 
objectives to ascertain the degree to which the objectives 
have been achieved. Thus they are only meaningful If they 
relate to the stated objective to be measured and can be 
compared with a realistic and attalnajle benchmark. 

While the first three requirements could g^^nerally not be 
disputed, the final requirement of quantlf lability might cause 
some concern. The key word Is "measurable" and the Important 
factor Is how, precisely. Indicators can be "measured". In some 
cases, performance Indicators would have little or no meaning If 
only quantitative measures were allowed. Clearly a balance has 
to be struck between the relative convenience of quantitative or 
counting measures and the qualitative measures which have the 
potential to enrich, and help explain, these quantitative 
measures. A simple example should demonstrate this. A student 
receives a grade of 72% for an essay. What precisely does this 
mean? It means very little unless the mark Is related to other 
students' marks for the same essay question - that Is, the score 
has a benchmark of some kind. The meaning of the mark Is 
enriched still further If there Is Information available about 
the strengths and weaknesses of the essay as a piece of writing 
and the extent to which It addressed the essay question posed. 
I therefore suggest that qualitative as well as quantitative 
Information must be Incorporated In any comprehensive suite of 
performance Indicators since the former aids Interpretation 
wh^.d the latter enables comparisons to be made more readily. 
Moreover a number of measures need to be used because, as Roe 
and Moses (1986) suggest: 

. . . performance Is far too complex to be reliably or 
validly assessed on a points or any other quantitative 
system. Such assessments might entirely miss or 
misrepresent the living reality . . . because some of 
the Important factors In (organisational) performance 
are human and/or Intangible; residing, for example. In 
reputation. Influence, climate, morale, motivation, 
sense of direction, leadership,, originality, shared 
values and creative thought. Such assessments (are) 
therefore regarded as entirely unsuitable for 
bureaucratic decision-making. 

(Roe and Moses, 1986 p. 73) 
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Rot and Moses (1986) were, like others, alarmed at the potential 
misuse of Indicators. Their concerns Included that: 

Indicators could fall to capture the essence of what was 
happening; 

organisations would ♦-.allor their performance to fit the 
chosen Indicators, i., short, the Indicators could become 
strait- Jackets; and 

Indicators could be Imposed from above rather than 
negotiated. 

Performance Indicators can be used at various levels within an 
organisational tructure. In the New South Vales and Western 
Australian legislation the TAFE system as a whole Is reporting 
performance Indicators to government, but this Information can be 
put together or split up (and enhanced) In various ways at lower 
levels In the system. The hierarchy used might be the one shown 
In Figure 4. 




Figure 4 
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This hierarchy potentially begins at the level of the system and 
ends with an examination of the performance of Individuals. The 
Information Is In Its most consolidated form at the highest or 
system level. At lower levels It can be put together or broken 
down In various ways to suit the needs of that level In the 
system. 

Alternatively It might be that shown It- Figure 5. 




Figure 5 

While the first method of arranging the Information was 
concerned with TAFE's operational units (colleges, departments, 
regions etc.), the second Is concerned with the way In which a 
TAFE system organises Its educational or program provision. 
This may run across colleges, departments and regions and Is 
concerned with what TAFE does In its various courses and course 
elements. 
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The approach adopted for the development of suitable Indicators 
for a TAPE tystea Ig detemlned by the clrcunatancee encountered, 
the purpoiei of performance Indicators and the level of review 
required. 

PERPORHANCB INDICATORS. KVALDATION AND VALIDATION 

While the link Is not often made, the processes used In program 
evaluation and validation have considerable relevance to the 
measurement of performance. In the context of the Victorian TAFE 
system It has been suggested that program validation (a form of 
"summatlve" evaluation) Is outwardly focused and concerned with 
determining the worth of a program both In terms of the extent to 
which clients' needs have been met and uhe extent to which they 
have been satisfied with program outcomes; that Is, Is the 
program doing the right thing? On the other hand, program 
^VftlVtftton Is seen as Inwardly focused (a "formative" exercise) 
and places value on determining the worth of activities occurring 
In a program's process. The emphasis of evaluation. It Is 
suggested, could be rather more on the Improvement of worth and 
rather less as a judgement of worth. It alms to Improve the 
quality of program delivery and process (Making Changes 1986; 
Guthrie and Foyster 1987; Guthrie 1988). 

Both program evaluation and validation require a range of 
appropriate measuring processes which are formalised to the 
extent that they are built Into TAFE's operational procedures. 
Appropriate time, money and staff training needs to occur co 
ensure that such processes can work. Thus evaluations of various 
kinds together with program validations provide TAFE with 
measures of Its performance at the program level. Whole colleges 
may also be reviewed and performance Information gathered (see 
Byrne fit ^. 1984a and b). Individual departments might also be 
reviewed (McDonald and Roe 1984; Roe and Moses 1986; Roe ftt Al- 
1986) and a range of performance Indicators for departments has 
been suggested. Groups of teachers and Individual teachers may 
attempt to gather Information about their performance using such 
processes as action research, rllnlcal supervision and quality 
circles (Guthrie 1988a and b). The essential difference between 
many evaluations and the types of performance Indicators being 
discussed here, Is that evaluations can be "one-offs" to solve a 
short-term political problem or address a periodic need such as 
program re-accredltatlon. Other evaluations, and performance 
Indicators, involve an on-going process of information 
collection. Thus they are used more in a management role and may 
act as gauges on warning devices which monitor performance. 
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WHO OUFS THE PERFC»KANCB INDICAT(«S7 



Perfonunce Indicators can Involve more than Just "the system" In 
the declslon*maklng pro*^esses. Those within and outside the 
systei have an Interest In the measurement of performance. This 
Issue raises seve;:al Important Issues In my mind. First, who owns 
or Is responsible for the consequences of measuring performance? 
Therefore, who will collect the Information? Further, who Is 
entitled to view and to Judge what the performance Indicators 
show up? It might be argued that government, TAFE Itself and 
TAFE's various clients (Incl ding Industry, commerce and other 
elements of the community) have various rights both to view and 
judge. It raises the vexed question of, having measured 
performance what then Is done? The answer lies ultimately In the 
purpose of performance measures. Surely one of the major 
measurement outcomes must be quality assurance and the 
Improvement of performance. This, In the end. Involves people. 

Another Issue therefore relates to rathways for Improvement. It 
Is no coincidence that both figures 4 and 5 end with 
"Individuals" as the focus. Whether one starts wfth a system or 
a program area these are, nevertheless, merely artificial 
organisational units which conveniently group individual people. 
The failure or success of these organisational units is, 
ultimately, a reflection of the contribution made by those 
individuals (and/or the groups or individxials outside them), and 
the way in which the system la structured, given its purpose. 
The structure was created by people and can only be changed by 
them. The way in which the organisational units relate to each 
other also depends largely on the people within them. If 
improvement is warranted, it is not th^ systems that change 
themselves, but the Individuals within (and possibly outside) the 
systems who can facilitate or prevent change. 

The level of performance «nd its Improvement, therefore, depends 
on individuals contributing to the task at hand and the extent to 
which the Infrastructures they work within enhance or decrease 
their chances of performing well, or better. I will take up the 
^ssue of ownership again later. 

PERFORHANCE INDICATORS AND COST -EFFECTIVENESS 

As I have already suggested, cost-effectiveness is another 
performance .erm often used in TAFE. I believe that it is a 
component, ilthough a major one, of the wider concept of 
performance indicators. It is an acknowledgement that costs are 
an Important component in the educational equation; nevertheless 
it is clear that the efficiency and effectiveness of a system 
cannot be measured solely in monetary terms. Other indicators of 
efficiency and effectiveness are needed to complement information 
about costs to ensure, and assure, that the best use is made of 
available funds to achieve desired outcomes. Low cost without 
effectiveness is not a desirable aim for TAFE. 
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Is co«t-#ff#ctlv#ne«« Mttly an •fflclency measure, or It It 
tonethlng acre? Henderson (1987) has discussed the development 
of performance Indicators, particularly In Western Australia. As 
we have seen, other states as well as the federal government are 
moving In this direction In order to achieve more efficient, 
results-oriented management. As Henderson (1987) points out, 
both Western Australia and New South Vales have stated objectives 
of ^management performance Improvement' or. In oth«r words, they 
are trying to drive their TAFE dollar further. To that end they 
seek to measure both efficiency and effectiveness. Efficiency 
involves the management and utilisation of resources for the 
miixlmlsatlon of outputs with minimisation of Inputs (see 
Henderson 1987). Efficiency Is also Improved when output can be 
iiilntaln^d using fewer Inputs or when output can be Increased 
with a less than proportionate Increase in Inputs (Birch and 
Utcham 1985). 

Effectiveness may be regarded as the extent to which program 
outcomes achieve stated program objectives which have been set by 
some appropriate body. As Birch and Utcham (1985) suggest, 
something is effective if it achieves appropriate objectives (for 
example, objectives which are appropri^ce to the needs of 
society); it is efficient: If It achieves theise objectives with 
the optimal use of resources. Achievement of objectives Is the 
key. Level of performance Is also tied to the degree to which 
objectives are achieved. To me cost-effectlveress therefore has 
a notion of effectivenesa achieved efficiently . It Is not purely 
an efficiency measure. 

All this talk of effectiveness, efficiency and performance 
Indicators Is meaningless, however, unless there are appropriate 
benchmarks - standards against which performance can reasonably 
be Judged. To achieve certain outcomes there are (possibly) a 
number of alternate ways of doing something, each of which has 
attached costs. By calculating costs and measuring both 
efficiency and effectiveness the approach which represents the 
best way of achieving a particular outcome can be determined. 
Hence cost-effectiveness measures, amongst other performance 
Indicators of various types, can be used by managers to see how 
those things which are under their control are performing 
relative to each other. 
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FOUR FACTORS TO BE CONSIDERED IN DEVELOPING PERFORMANCE 
INDICATORS 



Four factors need to be considered In developing measures of 
performance. These are the financial, the educational, the 
administrative and the political. Performance Indicators also 
have to be suitable for various modes of program delivery - for 
example, on-campus, off-campus, on-the-job, off-the-job, and 
national or state/territory based programs. Each of the factors 
should be dealt with In turn. I shall start with the political. 

THE POUTICAL FACTOR 

The educational process Is an Inherently political one. TAFE Is 
no exception to this. Commitments to develop and deliver 
educational and training programs to meet the needs of 
particular groups are made by politicians and by others. The 
sensitivity of a partlcuHr program to, and Its Importance In, 
achieving the alms of government, the education system and other 
Interest groups lends weight to both the moral and pragmatic 
dimensions. If something Is Important morally (such as providing 
educational opportunities for socially disadvantaged groups) 
then resources will be committed to ensure results. It may be 
so political that the solution to the problem and the 
achievement of outcomes may be Judged by very different criteria 
to those that normally apply. While political Issues may not 
directly affect performance Indicators, they may have the effect 
of changing the balance between the moral and the pragmatic. 
This, In turn, may change the benchmarks or criteria by which 
performance Is Judged thus making the whole process a political 
one . 

The processes of Identifying educational and training needs as 
well as developing. Implementing and evaluating programs and 
courses based on those needs Involves political processes as 
well. There are political forces at work which act between: 

the TAFE system and external Interest groups (e.g. 
government groups representing the Interests of Industry 
and commerce, accreditation authorities, licencing boards, 
training committees etc.); 

the various external Interest groups themselves; 

various levels within the TAFE system (for exfjsple "Head 
Office" and the colleges); 

colleges or other grou^a Involved, or likely to be 
Involved, In Issues concerned; and 
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(occasionally) between TAFE systems themselves. 

Political Issues may arise from the failure of Individuals to 
negotiate or even Just get on with each other and this may lead 
to the Issue escalating out of all proportion to Its real 
magnitude. 

Thus every aspect of educational enterprise Is actually, or 
potentially, a political arena. It Involves the exercise of 
power to control or Influence process. The proposal to change 
something very often leads Zo all sorts of ways being found to 
subvert or otherwise avoid making any changes at all (Guthrie, 
1988c). 

Jim Hacker (Lynn and Jay's creation In Yes Prime Minister) 
describes things this way: 

. . . the point about government Is that no one has 
control. Lots of people have the power to stop something 
happening - but almost nobody has the power to make 
anything happen. We have a system of government with the 
engine of a lawn-mover and the brakes of a Rolls Royce. 

(Lynn and Jay (eds.) 1986, p. 140) 

TAPE'S processes, Including curriculum development and review, 
might be described In much the same way. In fact, In a 
quotation attributed to Machlavelll (which was found pinned to a 
wall In the offices of a particular TAFE authority) It Is 
suggested: 

There Is nothing more difficult to take In hand, more 
perilous to conduct or more uncertain In Its success, than 
to take the lead In the Introduction of a new order of 
things, because the Innovator has for enemies all those who 
have done well under the old conditions, and lukewarm 
defenders In those who may do well under the new. 

Since everything else In education potentially has a political 
agenda. It Is not surprising that the Issue of performance 
Indicators does also. Moreover, If this Issue Is resolved and 
Indicators are to be, or are being, used the political agendas 
can swing to: 

what Indicators are being used; 

how they are being used; and 

how valid they are. 
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The balance achieved between the potentially competing purposes 
for using performance Indicators may Itself give some Indication 
of the political forces which have been at work to shape their 
purposes and use. 

In fact, there are a large number of political Issues and 
questions which surround the development, Implementation and use 
of performance Indicators. A range of these questions will be 
presented later when I consider at least some of the Issues and 
concerns raised by performance Indicators. 

THE FINANCIAL FACTOR 

This factor Is concerned with the direct and Indirect costs of 
providing education to achieve particular outcomes. The 
financial factor also Is concerned with: 

the costs which are actually attributed to the program and 
the ability of the accounting system used to calculate the 
costs of programs and program elements; 

whether the calculation made can be defended. 

Moreover, the financial factor Is dictated In part by the degree 
to which existing TAPE infrastructures and boundaries are 
regarded as sacrosanct. For example, some sectors of TAFE's 
operations (e.g. the provision of distance education) have the 
potential to be truly national in character. National 
co-operation in the development and delivery of distance 
education programs and materials is possible to achieve real 
economies of scale. In Australia there are a number of examples 
of such co-operation already. However there are many 
essentially artificial barriers to such co-operation as well. 
Other factors which need to be borne in mind when considering 
the financial aspects of cost-effectiveness Include: 

development costs (these tend to be relatively higher for 
off-campus study than for face-to-face and Include academic 
and materials production. Nevertheless distance education 
materials are often used with great effect in the 
classroom) ; 

the cost of the chosen delivery systems (a range of 
alternatives may need to be considered. The costs Include 
development and distribution of resources, methods and 
amount of student- stuc^ent £.nd student-staff interaction, 
the type of student activities used and assessment 
procedures) ; 

economics of scale (these not only Include the enrollment 
numbers needed to reduce unit costs considerably, but also 
the range of offerings - the greater the range offered, the 
greater the expense); 
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the type of course offered (science and technology -based 
subjects and courses are likely to be more expensive to 
mount and nin. Courses Involving significant practical 
work are also relatively expensive); 

. the cost of program or activity review (that Is, the cost 
of gathering the data needed for performance Indicators 
themselves) ; and 

the "proper" costing of programs (which may not only 
Include the direct costs of the program, but also capital 
or equipment replacement costs, as well as Infrastructure 
costs for facilities such as administration, libraries, 
personnel departments, student counselling services etc. 
However It may not consider costs which are actually 
Incurred by the system but, for various reasons, are never 
actually charged to a particular program or activity). 

Thus the accounting or financial system used needs to be able to 
measute or estimate costs of various components of the TAFE 
system and assign these costs appropriately to particular 
programs or activities. It also needs the ability to allocate 
Income received from students and other sources to various 
programs. All of this presents a complex accounting exercise, 
given the diversity of TAFE^s operations. 

THE ADMINISTRATIVE FACTOR 

Face- to- face teaching may Involve a high capital outlay In 
buildings and plant. The costs of salaries alone typically 
consume 70-80% of recurrent budgets. On the other hand, 
programs offered off- campus have relatively high administration 
costs In terms of their distribution and student support 
systems, as well as their production Infrastructures (Ashenden, 
1987). However there Is a much smaller capital Investment In 
buildings because there Is little or no need for large numbers 
of classrooms and other physical facilities. Nevertheless, real 
costs are Incurred If TAFE colleges or other community 
facilities (schools, community halls etc.) are used as part of 
the delivery mechanism. Sometimes the real cost of using 
facilities remains only a "notional" cost because these costs 
are never actually calculated. 

There are other administrative costs too. For example, student 
record systems, assessment processes and other administrative 
aspects associated with delivering attd running the services 
provided. Much of this can now be done by computer management 
systems. However these costs need to be applied to particular 
cost centres. Again, this Is a relatively complex accounting 
exercise unless * rules of thumb' are used. TAFE's £idmlnlstratlve 
Infrastructures also need to perform well - they need to be 
cost-effective. Providers (whether face-to-face or distance) 
have to reach certain enrolment thresholds In order to achieve 
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reasonable economies of scale In administrative and other 
costs. Smaller colleges are claimed to cost more to run than 
larger ones (Newell 1987) and so the notion of the moral 
obligation to provide a particular service Is used to justify 
their continuation. 

HIE EDUCATIONAL FACTOR 

The educational factor is concerned with the educational aspects 
of preparation, delivery and review of programs or activities. 
Education, of course, as the "nitty- gritty" of the enterprise - 
the administrative and financial factors are support mechanisms 
for education, which is the real focus. The educational factor 
Itself has two primary foci - needs (and their related 
educational objective^) and outcomes. Effectiveness (as I have 
already said) is the extent to which the objectives have been 
achieved by the measured outcomes. Appropriateness is measured 
by relating objectives to the actual needs (see Figure 3). In 
the present context the educational factor is most concerned 
with effectiveness and its measurement. However, knowing what 
the needs really are and what outcomes have been achieved are 
important features in any effective system of performance 
Indicators. Suitable indicators of effectiveness have to be 
found and related to measures of efficiency • including costs. 
What, then, are some of these measures of effectiveness and 
efficiency? These will be considered in the next section. 
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SOME PERFORHANCE INDICATORS 



INTRODUCTION 

There are many lists of performance Indicators, some of which 
will be referred to here. Others are Included In the references 
cited In the Bibliography. While many of the Indicators appear 
to measure the same Item (say student withdrawal), the figures 
they yield may not be comparable unless the definitions are 
substantially the same. For example, some Institutions may 
consider a student as withdrawn only when that student has 
formally withdrawn from a program; others may co\int those 
students who have functionally withdrawn even though there has 
been no formal withdrawal. Thus comparing apparently similar 
Indicators requires caution - It can be a trap for the unwary. 
This Is particularly so when a variety of techniques for program 
delivery Is used. 

The range of performance Indicators Is potentially very large. 
Those chosen depend on: 

• the lev el at which the performance Indicators are to be 
used (l.e, at the syst m level on the one hand, an 
individual teacher on the other) ; 

. the appropriateness of the 1 idicator given accountability, 
management, or other needs - tcr example, whether the 
indicators are properly focused on the information needs of 
TAPE clients or TAFE providers: 

. their co ntribution to a comprehensive package of 

performa nce information - that is, the indicators chosen 
should provide a full and comprehensive indication oi 
performance as possible; 

. the objectives of the orga nisation - the indicators chosen 
need to reflect th' system or organisation's objectives in 
order to measure its performance validly; 

. .the ability of the or ganisation to collect the Information 
- this relates to the personnel, time and money available 
to a system or organisation for collecting, processing and 
reporting information. 

Lists of performance indiciitors are contained in the 
publications of the Further Education Staff College in the 
United Kingdom (the Coomba Lodge reports) well as those of 
the National Center for Research in Vocational Education a Ohio 
State University in the United States. A number of these 
publications is cited in the Bibliography. 
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Sone work has been done at the tertiary level In Australia 
funded by the (then) CTEC Evaluation and Investigations Program, 
for exaiflple Bourke (1986) and Roe and Moses (1986) or by a 
working party set up by the AVCC and ACDF. Other work has been 
conducted In the public sector • for example the Program 
Evaluation Bulletins of the Public Service Board of New South 
Uales (see Bibliography). Individual TAFE systems In Australia 
are beginning to develop an active literature on the design and 
use of performance Indicators. There has been a limited (but 
growing) sharing of expertise between authorities. The 
Bibliography In this monograph therefore provides references to 
some of the work reported both here and overseas. The 
references cited cover performance Indicators for systems, 
colleges and organisations, departments, programs and 
Individuals. 

Earlier I referred to a meeting of people from various TAFE 
authorities which produced a list of suggested performance 
Indicators (TAFE Performance Indicators Planning Group 1988). 
This list was by no means exhaustive, nor was It Intended to be. 
Nevertheless It was agreed .zhat the list was a good starting 
point and that the Information about the Indicators proposed 
could be collected and would be useful at least at the system 
level. The list Includes two types of measures - those concerned 
with efficiency and those concerned with effectiveness. The 
measures which are described below were drawn from a paper 
presented to the Conference of TAFE Directors. Other measures 
of efficiency and effectiveness are suggested In the references 
presented In the Bibliography. 

EFFECTIVENESS INDICATORS 

These were: 

Graduate success rate - a potentially useful measure of the 
effectiveness of the match between TaFE courses and labour 
market requirements. I suggest that the Information could 
Include data on their employment status - for example: 

- number employed (full-tlme/part*tlme) ; 

number employed at an appropriate area given their 
qualifications ; 

number employed at an appropriate level given their 
level of qualification; 
number who are unemployed; 

number employed outside their area of training; 
number Involved In further study; 

. Student satisfaction with course provision - a global 
measure of students' experiences of a particular course. 
This Indicator has a number of potential dimensions, 
Including the effectiveness of course provision (Were 
courses and electlves available at appropriate locations?), 
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course content, levels of knowledge and skills gained, 
physical and other resources provided for the course etc.; 



Sat examination rate - a proxy measure of retention rate; 

Eyamlnati on pass rate - an indicator of the level of 
difficulty of a course and its individual subjects and a 
measure of successful skills acquisition; 

• Participation rat e bv demographic yroup - a measure of the 
degree to which TAFE is meeting the needs of its targeted 
groups and/or the social justice objectives of governments; 

Industiv'^ satisfaction - this indicator is manifested by 
the deg"*ee to which TAFE programs are meeting industry's 
perceived needs and the extent to which TAFE graduates are 
seen by industry as having the skills and knowledge to do 
the work for which they were trained; and 

Annual g raduate number - a measure of the number of fully 
qualified TAFE students available to the labour market each 
year. 

Another measure might include the level of potential demand 
actually being serviced. This would require the integration of 
enrolment and graduation data with any available data on labour 
force planning. Naturally there are many other effectiveness 
measures that might be used. 

EFFICIENCY INDICATORS 

The efficiency indicators proposed include: 

Average class size - this indicator is a means of 
monitoring the efficiency with which TAFE uses its 
lecturing staff; 

. Average hours taug ht in dutv time bv full-time TAFE 

lecturers - another means of measuring the efficiency of 
staff use; 

RecurrAnt rQfi^ per subject per student - a means of 
assessing the cost-efficiency of program delivery in a 
variety of lites, courses, study modes etc. The cost could 
a\30 be calculated in relation to successful students, 
surviving students, enrolled students, or in terms of 
student contact hours; and 

Sti.dent contact hours per seat of student capacity - a 
crude initial measure to assess the efficiency with which 
TAFE facilities are being used. 
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Other efficiency measures might Include unit costs, which could 
be broken do%m by region, college, department, delivery system, 
course, subject or year. The use of college, course or subject 
resources and facilities could also be estimated and monitored. 

The revenue earned by the system, college or departmt^nt could 
also be calculated as a ratio of the total revenues received. 
It Is obvious, therfore, that there are a large number of 
efficiency measures available for use. 

Each of the Indicators described can be split up or combined In 
a number of ways to provide Information for appropriate groups - 
Including various levels of management within TAPE'S structures. 
While a number of potential indicators have been Identified, 
there are a number of Issues and concerns raised by their 
definition, development and use. These Issues and concerns will 
now be considered. 



P^iRPORHANCB INDICATORS - SOKE ISSUES AND CONCERNS 



SOME QUESTIONS ABOUT PERFORMANCE INDICAiX)RS 

Pertonnance Indicators are a hot Issue In TAFE at the moment. 
There Is considerable pressure from many quarters to see them 
devisloped, Implemented and used. In particular, Impetus has come 
(as we have seen) from both the federal and state governments as 
part of their respective drives to Improve efficiency and 
effectiveness In the public sector. It has also come with the 
adoption of program budgeting by many government departments and 
agencies. However there are many who have reservations about 
performance Indicators. These reservations stem from their 
concerns that Indicators will become strait- Jackets which will 
hamper rather than help TAFE's operations. Given that 
performance indicators are Inference measures, a series of 
questions for as. casing their relevance and validity might well 
be asked. I have tried to provide some background discussion 
about each. The questions Include: 

Can performance I ndicators validly represent reality ? This 
question will be considered in some depth shortly. 

Will mor e mean better ? There is real concern that the 
development and use of performance indicators is directed 
solely at Improvements in efficiency - in other words . 
enroling more students, increasing class sizes, using 
learning resources and other facilities more frequently. 
This problem stems from the fact that quantity measures are 
easier to develop and use than quality measures. The 
quality measures are seen as more intangible and concerned 
with issues of effectiveness as well as efficiency. It 
might be argued that TAFE should be more concerned with the 
quality of its activities and place less emphasis on 
quantity. Clearly the two need to be balanced, but there 
is a fear that the emphasis placed on the performance 
measures by various interested groups will result in an 
increased emphasis on the measurement of quantity and the 
demand for more using less. 

Will indicators be Imposed rather than neg^otiated ? One 
issue of concern to many is that performance indicators 
will be imposed from the top down, by government or other 
interest groups whether within or outside TAFE. It will be 
suggested shortly (when the question of "ownership" is 
considered) that the nature of the performance indicators 
needs to be negotiated between the provider and the users 
of the information. 
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Who has right of access to what information? This question 
raises the issue of the "right to know" of particular 
individuals and groups on the one hand as against the need 
to protect the rights of individuals and groups whose 
performance may be reflected in the indicators chosen. 
While people are, and should be, accountable, it is the 
consequences which result from tLe measurement of 
perfonrance which are raised here. One fear is that the 
knowledge gained from performance indicators will be used 
rather more to punish than to praise. 

This question can be resolved once a clear and agreed code 
of ethics and practice associated with using performance 
indicators exists. 

Will Indicators become str ait- jackets which dictate rather 
than guide decision-making? In this case it is conceivable 
that those whose performance is being measured strive only 
to achieve average scores on the "performance tests" the 
indicators impose. If the indicators chosen are poorly 
defined and narrowly focused it may lead those assessed to 
ignore important and innovative ways of improving 
performance because this Improvement will not be measured 
and therefore not be appreciated b; those to whom they are 
accountable. Moreover those whose performance is appraised 
may aim to gear their level of performance as close to the 
benchmark as possible rather than achieve the level of 
improvement they are really capable of making. In short, 
they want to preserve some fat to achieve next year's 
benchmarks. Unless real incentives are offered for real 
improvement, the only incentive that exists is to improve 
as much as is necessary to avoid any punitive measures. 

How will indicators be use d in decision-making and 
management processes ? This question relates to the purposes 
behind developing and using performance indicators and the 
balance struck between their potential roles in the 
management process. The issua of balancing the roles will 
be considered in some detail shortly. The other aspect of 
the question is the ef fectivenes.s of the the mechanism 
developed for using the performance information available. 
This is the "nitty-gritty" of iuiplementin; and using the 
performance indicators to achieve the defined purposes. 
The relevant concerns are: 

the balance struck between purposes; and 

the methods chosen to gather and use the performance 
information to achieve the purposes. 

Moreover there is a danger that the purposes may be stated 
in one way but, in fact, the indicators may be used in ways 
which are inappropriate to the stated purposes. 
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Do those who Interpret what the Indicators mean have access 
to adequate Information to make proper decisions? This 
question raise issues which include: 

How much of a certain type of information is enough to 
cake proper decisions?; and 
What balance needs to be struck between the 
alternative sources of information so that proper 
decisions are made? 

The question raises the issues of the validity of the total 
package of information available from all the relevant 
indicators. It is also concerned with whether the 
information is verifiable. That is, If one gave the task 
of measuring performance to another individual or group, 
would they come up with the same sorts of conclusions and 
suggest the same actions or decisions. The point here Is 
whether the information available from the chosen 
indicators is enoug h to make defensible decisions at an 
appropriate time although, with the benefit of hindsight 
people may never have enough information. 

All of these are important questions. They lead me to ask a 
range of further questions about performance indicators and 
their use. I have tried to be as comprehensive as possible In 
posing these questions. I will not only discuss briefly each of 
the questions raised, but also take up in greater detail some of 
the issues raised by the questions shortly. 

The questions include: 

What records and other data are routinely kept within TAFE 
systems (student, exaa, accounting etc.), and would these 
records assist in the development of appropriate 
performance indicators? How readily can these data be 
accessed, consolidated or upgraded (if necessary) to meet 
the needs of a performance indicator system? These 
questions relate to the data bases which a TAFE system 
maintains at its various operational levels. The first 
steps in answering these questions are to establish what 
the chosen indicators of performance will be and how they 
will be measured. Given this, the next step is to 
establish what relevant information is, or should Le, 
already available and its sources. This raises the issue 
of the compatibility of the various data bases on which the 
information needed Is held and the extent to which this 
information can be accessed and manipulated to obtain the 
indicators required. It is tempting to let the available 
information determine what is needed. This temptation 
should b i resisted. 
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What benchnarks for indicators are appropriate? To what 
extent do indicators for various nodes of delivery have to 
be coaparable? Who sets tb3 criteria for •success"? 

Obviously any chosen benchmarks have to be reasonable and 
take appropriate account of the moral and pragmatic 
dimensions of TAFE's activities. To be appropriate, those 
Involved need to agree on the benchmarks ; that is , what 
will be the agreed criteria for "success". The indicators 
chosen have to be fair and take account of alternative ways 
of achieving outcomes. They should not be biased to 
reflect favourably on a limited range of alternatives. 

Which Indicators of performance would be valid and 
useful? What are appropriate definitions for any selected 
indicators? Are these definitions broadly applicable: 

across nodes of delivery? 

- within individual states or territories? 

nationally? 

The question of validity is considered shortly. The 
usefulness of particular indicators will be determined by 
the need to know. This is a management decision which 
could centre on likely areas where performance, 
particularly poor performance, would be a critical issue. 
The decisions about which indicators are useful and valid 
are determined by the information particular groups and 
individuals within or outside the system require to help 
them to make decisi'^'^s and Judge quality. The 
appropriateness of indicators depends, therefore, on the 
balance of purposes and the ways in which the information 
is used. Whatever happens the definitions used must be 
agreed to ensure that indicators are verifiable by 
independent audit if required. Again, to be fair to all 
parties concerned, definitions have to be applicable across 
modes of delivery and certainly within states. It is 
arguable that the definitions of agreed key indicators 
should be common nationally to ensure that any comparisons 
made between TAPE systems (for example) are valid. 

How will the indicators be collected? How will records be 
maintained? These questions clearly have to be addressed 
during the development and implementation of any systematic 
gathering of information. Whatever method is chosen it 
needs to be cost-effective and those helping to collect and 
maintain records need to agree that theirs is a useful role 
and that adequate time and resources have been allowed to 
enable the information to be collected. 
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Hov vlll indlca'' jrs be used in tJie decision-making 
processes? 

- St i '•^jrritory level? 

- nstlcf.iAlly? 

Again, this question needs to be addressed during the 
process of developing and implementing indicators. At the 
state/territory level their use in decision-making is most 
likely to be in discussions between government and TAPE on 
the provision of resources, and between TAFE's central 
office and individual colleges and other providers over the 
level of resources they will receive. Clearly the reso^^rce 
allocation will be agreed in terms of the negotiated anu 
agreed benchmarks. 

A similar argument would apply at the national level with 
state governments, TAFE systems or eve^' individual 
providers negotiating with federal authorities over 
resource allocations based on agreed performance ^ vels and 
method of measuring performance. Within TAFE systems, and 
particularly at the college or program levels, the 
indicators may be less tied to resource agreements and usv 
rather more as means of quality assurance and identifying 
ways of making operational improvements. The decision* 
making process does not necessarily have resource 
implications . 

To %ihat e;^tent would performance indicators be *^fjd for 
internal management puiposes as aguinst external 
accouTitability requiremencs? This question will be 
considered in greater detail when I consider how the 
various roles of performance indicators could be balanced. 

Hov "^^ght the internal usefulness of indicators be 
maxii>^sed? The usefulness will be maximised if those 
within TAFE systems agree with the performance indicators 
chosen and the methods by which the information is gathered 
and used. Ownership of the process will enhance their 
utilization. Ownership also implies that there has been 
negouiation and agreemint between the various providers and 
clients of TAFE's serviced. 

Hov might social Justice indicators be developed and 
gathered? Are these needed? Hov valid vould they be? 
The issue of social Justice Indicators is difficult to 
address. Certainly such indica\ ors should be developed 
because social Justice is an important part of TAFE's 
mission. The mix of TAF*E students can be examined to 
determine what groups are participating and their 
proportion given their size. This participation, however, 
should be related to the types of programs in which they 
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participate - for example, does a particular disadvantaged 
group participate In access programs, but then never 
actually Join mainstream programs or gain employment? 
Setting benchmarks and devising valid measures which 
address social justice Issues Is difficult. In allocating 
resources an appropriate balance also needs to be struck 
between moral and pragmatic demands. Where this point 
should lie Is always open to considerable debate because 
the cost of delivering services to certain groups Is often 
out of all proportion to the benchmarks which are normally 
accepted. 

Hov do ve avoid Baking Indicators strait- Jackets? The 
danger la that Indicators may become rather less indicative 
and rather more prescriptive. This Issue of performance 
Indicators becoming strait-Jackets has already been 
discussed at some length (see page 31). Negotiation 
between the client and the provider of the performance 
information should help avoid prescription and depends, in 
part, on a comprehensive set of Indicators being employed 
which does not merely concentrate on the easy and 
convenient, but rathet' attempts to obtain measures which 
are valid and useful to all concerned. 

What system of penalties or incentives, if any, should 
exist for individuals or groups wh.o exceed or fall to meet 
targets? There is little point in measuring performance 
if no use is to be made of the information. While it is 
arpued that the major use of such Indlctors should be self- 
regulation, there is little doubt that both penalties and 
incentives in some suitable form are required. There need 
to be Incentives to encourage people and groups to rate as 
high as possible and their achievement should be recognised 
in some way. Penalties should be Imposed too, but only on 
the basis that a better performance was undoubtedly 
possible given the circunistances. Failure to meet 
benchmarks or targets mav merely be a reflection of the 
unreasonableness c£ those targets in the first place. 

Hov do we wold overloading managers, and others, with 
Jri^omatlor? When information becomes available, there 
can be a teiptation to provide too much rather than too 
li*"tle. Clearly managers need training on how to Interpret 
aid use the 1-iformatlon performance indicators provide. 
However considerable attention should be paid to the way in 
which the collected information is processed and reported, 
so that the manager is given a balanced view, but 
nevertheless sufficient information for the purposes of 
dec is ion- making. Achieving this balance is rather more of 
an art fonr than a science 
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Hov Buch tlM, effort and money will the collection of 
perfonance Infornatlon cost? If little or no eiiort has 
been put Into gathering performance Information previously, 
additional time and other resources will be needed Initially 
to set up and run a system for gathering performance 
Information. 

Those developing and using such a system need to en^^ure that 
the system chosen Is effective, whilst also being 
efficient. It needs to be tempered with pragmatism; there 
Is a lav of diminishing returns whereby an Improvement In 
the accuracy of available Information can only be achieved 
with much greater cost of time, money and effort. The 
system has to be seen to be worth what It costs once the 
collection of Information becomes more routine. 

Vhat ethical Issues, if any, are Involved In the collection 
and maintenance of performance Indicators? Tliere an 
clearly ethical Issues Involved In the collection and 
maintenance of performance Information and these have 
already been discussed to some extent earlier. If management 
decisions are made, and If rewards or penalties are Imposed 
which are subsequently shown not to be wrranted, then 
considerable doubt must be cast on the validity and 
reliability (verlf lability) of the Indicators used. 

'*eover It may not be the Indicators themselves that are at 
fault, but the way In which they have been Interpreted and 
.sed. There are also ethical Issues Involved If an 
Individual's career and opportunities can be unjustly 
affected through their use. Some code of practice Is needed 
which balances the protection of peoples' rights with 
requirements for full and frank disclosure of Important 
Information. 

Hov will any activities to develv^ performance Indicators 
In particular territories and states be Integrated with 
other, similar developments In TAFE at the state/territory 
and national l«»vels? Most TAFE systems In Australia have 
already developed, or are beginning to develop, performance 
Indicators. The amount of Information exchanged both 
formally and Informally between states Is heartening. It is 
clear that such a process will continue, and that the merit 
of developing and using key Indicators based on nationally 
agreed definitions and methods of gathering Information has 
been recognised. 

How do ve avoid biasing the Information collected, given 
that certain Infornatlon Is more readily collected? This 
question will be considered In greater depth shortly. Any 
suite of performance Indicators needs to be as comprehensive 
as possible. Efficiency and effectiveness measure^^ alike 
need to be developed and, at certain points, the 
appropriateness of the chosen strategies and activities 
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haft to be Judged In the light of the "real** needs. In this 
way the Information collected should not be biased. The 
chance of bias Is Increased when only readily available 
measures of performance are used. Such measures are likely 
not to be comprehensive. Bias is also reduced If there are 
agreed procedures for gathering, processing, Interpreting 
and reporting the performance Information. 

Who control < the selection of the information to be 
gathered? Who *ovna* and has control over the ways in vhich 
the information is gathered and reported? These questions 
are considered shortly when I take up the Issue of ownership 
of the process. 

What account, is taken of performance indicators vhich 
measure long-term success as against those which measure 
short-term success? While this Issue will be considered In 
more detail shortly it Is clear that any valid measures of 
performance concerned with effectiveness need to take 
account of both the long-term and short-term measures of 
"success". However there is a problem of assigning 
responsibility for whatever success has been achieved. 

Vho is responsible for, or has contributed most to, the 
successful achievement of outcomes? This question will be 
considered when validity is discussed. 

How readily can information collected be combined or split 
tip? The degree to which information can be combined or 
split up will depend on the data base or bases used and 
their compatibility. Moreover, if the system has been 
designed purely to obtain, process and output specific 
information, it is possible that it will be relatively 
inflexible. To use performance indicators to maximum effect 
requires the caieful drafting of system specifications so 
that they retain the maximum flexibility should presently 
unforeseen information needs arise. 

These and other questions touch on vital issues of concern to 
those involved in measuring performance. 

PERPORMANCE INDICATORS - SOKE ISSUES OF CONCERN 

The questions posed above raise in my mind at least, a number of 
key Issues requiring mere detailed discussion. These issues are: 

the ownership of the process; 

the bAlancing of the various roles of the process (i.e. the 
translation of theory in^o practice); 

the collection of information; and 



the validity of the process. 



Naturally these Issues are clearly linked, but I will try to 
discuss them Individually. 

Ownership 

I have already briefly considered the Issue of ownership of the 
process when performance Indicators were first defined and 
discussed. The questions raised Included: 

Who owns, or Is responsible for the consequences of 
measuring performance? 

and 

Who Is entitled to view and to Judge what the Indicators 
reveal? 

These questions are difficult to answer generally. The situation 
will vary from state to state and be affected by the legislation 
In place, the ^ ^anlsatlonal structure rZ the system and the 
extent and ways in which performance Indicators are used In that 
system. 

The ownership of the performance Indicators Is very much tied up 
with the balance between Its functions. The legislation In some 
states Imposes a "top-down" demand on TAFE systems to provide 
information about their performance. It is an accountability 
function, but undoubtedly it can also be used in government 
planning and also in managing resource allocation. On the other 
hand, a "bottom-up" process exists where managers at various 
levels in TAFE require performance information for accountability 
purposes and for planning, decision-making and quality assurance. 
If, ultimately, the process of top-down accountability is 
intended to introduce processes of self -regulation, then the 
chances of TAFE owning and using the performance indicators are 
enhanced. This likelihood is further enhanced if the TAFE system 
commits enough resources to develop an appropriate and 
comprehensive system for measuring performance and ensuring that 
the system works. For effective implementation, the task of 
collecting and reporting performance indicators cannot be an 
additional burden which is imposed on a system which may already 
be overstretched. In this situation there have to be trade-offs 
if the maintenance of an effective system for measuring 
performance is felt to be important. 

Moreover, a dihtinction has to be drawn between ownership of and 
interest in the processes of measuring performance. While various 
hierarchical levels may r^^>'ire the collection, processing and 
reporting of information relating to performance, they may only 
require it from time to time. If they are not working with this 
information regularly they could be said to have an interest in 
the information collected, rather than "ownirg" c. 
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Moreover those higher In the hierarchy depend on the goodwill of 
those In lover levels In the system to provide this Information 
gathering service. Thus, to enhance opportunities for getting 
useful Information the task of gathering must not be a chore - 
there must be something In It for the gatherers and users - for 
the real owners. 

Too often systems for monitoring performance ar;^ Introduced with 
little understanding of the Implications. Information does not 
collect Itself, and to Impose additional work on a system which 
sees Itself as overworked or threatened Is a recipe for 
disaster. To Improve ownership, therefore, layers of Indicators 
are required which, together, make up p. useful data base for 
measuring performance and which satisfy the needs of all those 
who require such Information and, ultimately, those who 
legislated that certain Information should be provided. 

The two main groups Interested In ownership are the clients and 
the providers. Theodossln (1987) describes clients as those who 
make demands, and providers as tho.'>e who attempt to satisfy 
those demands. Clients clearly Include students; Industry; 
commerce; government and semi -government organisations; TAFE 
administrations; particular social, ethnic and other community 
^roups (Including support services); Industry training 
committees; college directors and their councils; regional 
managers; heads of departments and licensing authorities. This 
list, of course. Is by no means exhaustive. 

Providers encompass a wide range of Individuals and groups. 
Including TAFE systems, administrators, college directors, heads 
of departments, teachers, support staff, etc. Needless to s^y 
particular groups and Individuals can be both providers and 
clients. Their role depends on the context. Their view of the 
world and their Information needs will be different depending on 
the role they are playing at that particular time. Thus It Is 
not surprising that the question of who owns the performance 
Indicators Is a difficult one. The fact Is, the process of 
gathering performance Indicators Is not a process, but many 
processes which, potentially. Inter- relate. It Is Important to 
resolve ownership questions to avoid the waste of resources and 
to devise an Information gathering system that enables 
Information to be aggregated, disaggregated and assembled 
efficiently according to the needs of particular clients. Some 
of the clients are outside government and the TAFE system. 
Their ownership, as I have said. Is not a constant one; they may 
only want to be Involved occasionally when critical Issues 
present themselves. Their right to know Is therefore difficult 
to determine because they are often outside any recognised 
management structure. To say that the processes Involved In 
gathering and Integrating performance Information are complex 
and difficult Is to understate the problem of the right of 
access to Information. To work best the system requires the 
agreement of many parties to ensure that the necessary 
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information Is available. There must also be clear guidelines 
about what Information be made available and there are, as 
we have seen, considerable ethical Issues Involved. 

Balancing th« roles 

Performance Indicators clearly have a role as accountability 
measures. This cannot be denied. Nevertheless Individuals can 
be held accountable without any serious attempt being made to 
evaluate their performance (McTaggart 1988). Thus 
accountability can be about exercising power, and performance 
Indicators can be used to provide the means to exercise that 
power. On the other hand, I have suggested that performance 
Indicators have a role In the management, quality assurance and 
decision-making processes. Again, the effectiveness of the 
performance Indicators used In these roles depends on the value 
and the validity of the chosen Indicators. 

In management they can be used to hold groups or Ir 'Ividuals to 
account for their performance - possibly using Internally 
Imposed criteria. As I have already suggested, this management 
style, although It might work, can tend to focus on failures in 
performance, rather than providing a balanced view of the 
performance which combines elements of good performance with 
those where improvements might be made. 

If the demand for performance indicators is rather less for 
accountability purposes and rather more for self -regulation 
then, as I have already suggested, the problem of ownership of 
the performance Indicators will be lessened. People do not want 
to teach a poor course; people do not wish to see money wasted 
on facilities that are not really needed. Many would like to 
find ways in which they could improve, or their department, 
college or TAFE system could become more efficient and 
effective. Balancing the role of Indicators should emphasise 
the management role to assist with decision-making and to ensure 
that the quality of TAFE's activities and operations is at least 
maintained, if not enhanced. On balance the accountability role 
should be played down 

Collect ing the information within a TAFE system 

A comprehensive system of performance monitoring requires that 
information be collected at a variety of levels within a TAFE 
system; this information may then be consolidated at higher 
levels within the system until it is reported externally to 
various Interested clients. Clearly in collecting Information 
the requirements of both clients and providers need to be 
identified and balanced appropriately. 

A comprehensive, but cost-effective means of gathering and 
reporting information is required. What is needed is a system 
which enables an organisation not only to fulfil its obligations 



43 

47 



(that Is. be accountable) but also to defend Itself against the 
possible consequences of ill-informed decisions made on the 
basis of inadequate information. This applies not only at the 
system level but also at other operational levels. 

As data are progressively consolidated, distinctions between the 
various subgroups can become blurred. Thus poor and exceptional 
performances alike become disguised, unless the poor performers 
are large and swamp the better performances of lesser groups. 
Therefore, as information is combined there is always the danger 
that its richness and variety will be lost. Making decisions 
using these combined data and their associated benchmarks or 
formulae can be unfair and demonstrate a lack of sensitivity to 
the range of prevailing circumstances. 

Who collects this information? A range of people will clearly be 
involved. TAPE records are maintained by administration staff 
based in colleges and centrally, and these databases will also 
contain much of what is needed. The potential problems in the 
short- to medium-term include the compatibility of various data 
bases and the extent to which the data within them can be 
combined and manipul:it«»d to produce the desired performance 
measures. This can be overcome if the information system is 
flexible and capable of interrogation. There is a problem, 
however, if an information system is designed merely for the 
here and now. 

lecturing, curriculum development and research staff also have a 
role to play in gathering, processing id reporting informati/n. 
They are likely to be most interested in the indicators of 
program effectiveness and appropriateness. However, when 
comprehensive reports on indicators are required, the most 
likely problem is that people at various levels within the 
system will be unaware of what information is available 
elsewhere and how it can be accessed. There may be concerns 
about how effectively the information can be used, given that 
many within TAFE have had little or no experience in collecting, 
processing and evaluating information about performance. 

Moreover, definitions of particular indicators have to be agreed 
upon and applied consistently to ensure that the information 
collected is comparable between (for example) institutions and 
programs and that it can be aggregated validly. Those interested 
l^n the Yes Minister series may recall the occasion when a 
particular council had not sent in its statistical returns and 
was to be disciplined by Hacker's department even though it was 
the most efficient council in the U.K. Nevertheless information 
(although perhaps not the most valid information) had to be 
collected: 
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Sir Humphrey was quick to explain: 



*If local authorities don't send us the statistics 
we ask for, then government figures will be nonsense. 
They'll be Incomplete!' 

I (Hacker) pointed out that government figures are a 
nonsense anyway. No one denied It, but Bernard suggested 
that Sir Humphrey wanted to ensure that they are a complete 
nonsense. 

(Lynn and Jay (eds.). 1984, p. 493) 

Such a situation arose In the United States when the Department 
of Defence used body count (the number of Vietnamese dead) as an 
Indicator of Its military success (McTaggert 1988). Such a 
count was capable of such gross manipulation and falsification 
that It was complete nonsense. How much does this sound like 
TAFE's 111- conceived performance Indicator which Is referred tc 
In polite TAFE circles as "peoples' posterior parts on chairs"? 

The aim of having performance Indicators Is not to produce 
nonsense, complete or otherwise. Definitions and Indicators 
that i2 make sense and that useful and valid are what a TAFE 
system requires. 

The problem with measuring outcomes Is that many of them are 
nebulous, difficult to define and harder to measure - even 
indlrec;.ly. If a desired outcome Is to produce graduates who 
can solve problems, think creatively or transft.^ knowledge and 
skills from familiar to new situations, It may be difficult to 
measure the extent to which graduates have achieved those 
particular desired outcoi^es. Certainly, assessment activities 
can be devised which could test the achievement of those 
outcomes. Employees, the graduates' Immediate superiors and the 
graduates themselves might be able to provide Indications of the 
extent to which particular outcomes had been achieved. 
Nevertheless some outcomes take longer to achieve than others. 
Moreover some graduates might not be placed In a situation which 
tests their achievement of the particular outcome In the work 
place. Finally, If their TAFE training has occurred while they 
were working who Is to say whether the program or other aspects 
of their private life, their work or even their previous 
experiences prior to undertaking training contributed most to 
their apparent ability to (say) solve problems. In short, having 
observed the outcome, can It be said with certainty that the 
program, or TAFE, was largely responsible for that outcome being 
achieved? Thus It has to be acknowledged that performance 
Indicators are what they are - Indicators. They are gauges of 
performance and unusually high or low levels of performance 
might be questioned. It may be that the performance Is 
acceptable or, at least, explainable. It may also indicate, 
however, that there Is a performance problem which needs to be 
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addressed. Much of the real value of performance Indicators 
comes from using a variety of measures and In the valid 
Interpretation of their meaning. In this way the Interpretations 
made can be defended. 

The validity of the proceag 

Whenever Indirect measures are used people may question their 
validity. Moreover, It Is easier to gather data about some 
perform^.nce Indicators than others. It Is likely that these 
"easy" measures may be chosen even though they are biased and, 
together, do not constitute a comprehensive measure of 
performance. Others £.rgue that anything Is better than nothing. 
However, If the measures lack real validity there Is the strong 
possibility that the management and dec Is Ion -making processes 
served by the Indicators will be little better Informed than they 
were In the absence of data. Numbers have a beguiling face 
validity because we can see them and compare them, believing In 
their validity without really looking behind the facade to see 
whether the Indlcator(s) chosen have substance. As Henderson 
(1987) has recognised, there Is a distinct possibility that 
performance Indicators will be devised and then their 
Implementation approached from an unrealistic, bean-counting 
perspective, llie danger of this Increases as the Information Is 
progressively aggregated. Indicators are like Icebergs and 
managers have to ensure that what thev can see Is a valid 
lepresentatlon of what 1^. The system depends on having a number 
of measures of an aspect of performance which should help both to 
measure and explain the performance. The validity also depends 
on the benchmarks chosen. If these benchmarks are unreasonable 
and extreme no performance, however good, can measure up. In the 
end It Is people who define what Is, ana Is not, a reasonable 
benchmark. Thus the benchmarks used liave to be flexible and 
tempered by the twin demands of moral obligation and pragmatism. 

Furthermore, the performance Indicators chosen also have to be 
verifiable as well as valid (see Henderson 1987). The indicator 
is of little use if an independent individual or group using the 
same information cannot arrive at a similar answer or conclusion. 
To be verifiable it is necessary that definitions and ben'^hmarks 
for indicators be clearly stated. Moreover, the ways in which the 
indicators are calculated and data are combined or split needs to 
be documented appropriately. If the information used in the 
decision-making or management processes is not verifiable it is 
not likely to be defensible either. 

The quest for validity also has to be tempered by pragmatism. 
There are economies of scale to be considered and the need for 
validity needs also to take into account the costs of obtaining 
timely and valid performance measures. The question for 
administrators and those responsible for gathering information is 
- how much is enough? When is the improvement in accuracy of the 
information not worth the cost of achieving that level of 
accuracy? 
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Indicators also depend on the validity of the objectives, 
mission statements, or whatever is being used as a basis for 
determining effectiveness. If the objectives do not reflect 
needs (i.e. the objectives are inappropriate), or if the 
identified needs do not properly reflect the real needs, then 
the basis for making valid Judgements about performance is 
seriously threatened. Needs identification, occupational 
analysis and training needs analysis are all part of the 
curriculum research and development function. Course validation 
determines the extent to which the various needs were actually 
met. This forms part of a cyclical and continuous process of 
program development and review. The extent to which needs have 
been met and missions fulfilled is, as we have seen, extremely 
difficult to measure, yet measures of quality are of critical 
importance in terms of Judging TAFE's effectiveness (Henderson 
1987). 
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SOME CONCLOf^IONS 



This monograph has attempted to bring together some of the 
literature from both Australia and overseas related to the use 
of performance Indicators by TAPE. In Australia It Is clear 
that the major Impetus for the development and use of such 
Indicators has come from demands by TAFE's clients, principally 
the state and federal governments, to provide evidence that TAPE 
Is performing efficiently and effectively. The monograph has 
examined, and put the arguments both for and against using 
performance Indicators. It has also defined and discussed the 
term performance Indicator, particularly In the context of 
Improving efficiency and cost-effectlveness. Indicators not 
only have an accountability role but also a role In improving 
management and decision-making processes. They are also used as 
a quality assurance tool. 

From the available Information It seems that: 

performance Indicators will become an Increasingly 
important part of TAFE's methods of planning, operation and 
management given state and federal adoption of program 
budgeting; 

performance indicators can be Justified on theoretical, 
moial and pragmatic grounds; however, what makes them 
useful is the extent to which the good theory behind them 
can be translated into good practice in using them; 

the development and use of performance indicators is an 
Intensely political process and needs to be recognised as 
such; 

TAFE systems and tho:,e within them need to recognise the 
value for themselves in using performance indicators. If 
the indicators are not directly useful within TAFE there 
will be little or no incentive to own them. It will just 
be another chore ; 

whilst they are theoretically defensible, the ways in which 
they are designed and used may glv« rise to concerns about 
their validity as measures of performance. There is a 
danger that their users will expect more than the 
performance measures can actually deliver; 

the performance indicators chosen need to be adaptable to 
the variety of ways TAFE uses to meet its obligations; 
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th6r« will be Increasing pressure on TAFE systems across 
Australia to develop uniform measures of performance - 
given the possibility that performance measures which are 
similar but not comparable will, In fact, be compared; 

some TAFE systems have already done a considerable amount 
of work In designing, trlallng and using various 
performance measures. Other TAFE systems have only begun 
to explore the potential Inherant In performance Indicators 
and can therefore learn a lot from more experienced 
states. There Is a strong need to develop a network and 
dialogue between TAFE systems; 

If TAFE systems decide to develop and use performance 
Indicators they have to make a real commitment to making 
them work. Those who will collect, analyse and use the 
information need the time and expertise to collect and use 
the information judiciously. To impose performance 
measures as an additional load on inexperienced staff will 
ultimately be counterproductive. 

I am cautiously optimistic that performance indicators will be a 
beneficial tool for use not only by those outside, but also by 
those Inside TAFE systems. My note of caution stems from 
qu«»stions about the ownership and the validity of performance 
indicators and the need for appropriate methods of information 
collection. The optimism springs from the hope that, once TAFE 
systems have developed and begun to use performance measures, 
staff at all levels will be convinced of their value and that 
their use will lead to real improvements in TAFE's performance. 
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